DOCUMENT RESUME 



ED 284 497 

AUTHOR 
TITLE 

PUB DATE 
NOTE 

PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



HE 020 577 

Hobus, Pie P. M. ; And Others 

Contextual Factors in the Activation of First 
Hypotheses: Expert-Novice Differences. 
[86] 

13p. ; The research was supported in part by a grant 
from the Canadian Medical Research Council. 
Reports - Research/Technical (143) 

MF01/PC01 Plus Postage. 

*Clinical Diagnosis; Comparative Analysis; 
Educational Research; ^Graduate Medical Students; 
Higher Education; ^Information Utilization; *Medical 
Evaluation; ^Physicians; Problem Solving; *Recall 
(Psychology) 

ABSTRACT 

The role of contextual information in the generation 
of early hypotheses during the clinical interview was studied with 
expert and novice medical diagnosticians: 18 experienced family 
physicians versus 5 new physicians and 12 final-year medical 
students. The 18 experts and 17 novices were presented with 32 short 
case histories each presented on three slides; a portrait of a 
patient, a patient chart containing the previous disease history, and 
a slide with the present complaint. Because of the sequential nature 
by which patient data became available during a clinical interview, 
contextual information is expected to be important in the initial 
hypotheses. Differences in the number of correct diagnoses were 
predicted for the two groups, if experts used the contextual 
information, implicitly provided by picture and patient chart, in a 
more elaborate way. If this prediction was true, it would show in the 
amount of information explicitly recalled. The data confirmed these 
predictions. The experts produced 50% more correct hypotheses as 
compared to the novices and were able to reproduce a larger amount of 
relevant contextual information. In addition, a high correlation 
between problem-solving and recall measures was found only for the 
expert group. (Author/SW) 
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ABSTRACT 

According to Feltovich and Barrows (1984), the general frame used by medical experts to 
construct a cognitive representation of a particular patient problem, contains a component- 
part for those Illness features that are associated with the acquisition of illness. These so- 
called "enabling conditions" are recognizable in the context of the patient, like risk factors 
originating from work, behaviour and hereditary taint; or are derivable from patlenfs 
appearance like sex and age. 

Because of the sequential nature by which patient data become available during a clinical 
interview, this contextual information is expected to play an important role in the generation 
of initial hypotheses. 

To investigate the hypothesis, that experienced physicians better utilize this kind of 
Information, a group of 18 experts and 17 novices were confronted with 32 short case- 
histories each presented on three slides: a portrait of a patient, a patient chart containing the 
previous disease history, and a slide with the present complaint. It was expected that 
differences in the number of correct diagnoses would emerge between the two groups, because 
the experts use the contextual information, implicitly provided by picture and patient chart, in 
a mere elaborate way. If so, this would show in the amount of information explicitly recalled. 
The data confirmed these predictions. The experts produced almost 50% more correct 
hypotheses as compared to the novices and were able to reproduce a larger amount of relevant 
contextual information. In addition, only in the expert group a high correlation did exist 
between problem solving and recall measures. 
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INTRODUCTION 

Investigations into expert-novice differences in the ability to solve medical diagnostic 
problems are useful because they may suggest ways to teach this skill. Research in this area 
indicates that medical experts perform better than novices on a large variety of expertise- 
related tasks. They are better at diagnosing dermetological diseases presented on slides 
(Norman, Muzzin & Rosenthal, 1985), judging X-rays (Lesgold, Feltovich, 61aser& Wang, 1981) 
or recalling clinical cases ( Patel & Groen 1986). They are faster at processing patient 
information (Claessen & Boshuizen 1985) and show superior recall of texts derived from a 
medical textbook (Patel, HoPingKong & Mark, 1984). It is however, largely unclear which 
factors mediate this superior performance on these tasks. One of the reasons for this 
shortcoming may be that an articulated theory of expertise in this area, despite several 
attempts (Elstein, Shulman & Spraf ka, 1 978) is virtually absent. 

The purpose of this research is not so much to demonstrate differential abilities between 
physicians of different levels of expertise in yet another domain, but to look for reasons why 
these differences arise. In particular we were interested in finding an explanation for the 
known differences in accuracy of diagnostic hypotheses that emerge during the first moments 
of a clinical encounter. This expertise-related feature has our special interest because the 
accuracy of the first hypotheses to a large extend determines the accuracy of the final 
differential diagnosis (Barrows, Neufeld, Fe1ghtner& Norman, 1978) (Barrows, Norman, Neufeld 
& Feightner, 1982) (Neufeld Norman, Feightner& Barrows, 1981). 
Characteristic for the early stage of an encounter is, that apart from the complaint with 
which the patient confronts his physician, little information is given explicitly. A clear 
discrepancy exists between the small amount of complaint information and the large amount of 
information that is implicit in the situation. One such an information source is the patients 
non-verbal behaviour and appearance, another is the doctors prior knowledge about his patient. 
He knows the kind of diseases the patient has suffered from in the past, the drugs he uses or 
has been using, and the nature of his work environment. This kind of information provides a 
context within which a physician can search for a likely hypothesis explaining the patient's 
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present symptoms. Because of the sequential nature by which patient data become available 
during a clinical Interview, this contextual information Is expected to play an important role in 
the generation of Initial hypotheses. 

Feltovlch and Barrows (1984) have developed a theory that may apply to the present 
experiment. In their view basic science knowledge (e.g. pathophysiology, anatomy) plays an 
integral role in the construction of a cognitive representation of a particular patlenfs problem. 
It guides the way in which Illness features are structured together. 

The general frame which is used by the experienced physician to construct this representation 
consists of three component parts that are interrelated with each other. Illness features that 
are associated with the acquisition of Illness are termed enabling conditions. Instances of 
enabling conditions are predisposing factors like alcohol and nicotine abuse, or boundary 
conditions like sex and age. They provide relevant context of how the patients condition came 
to be. The second category of features: faults, contain tCie major malfunctions that lead to 
certain consequences. An example of this is the inadequate oxygen supply of myocardial 
tissue (which can be defined the fault), leading to anginal symptoms and eventually cardial or 
systemic complications (consequences). In the problem representation model of Feltovich and 
Barrows (1984) the previous mentioned contextual information can be classified as enabling 
conditions. 

In order to investigate the role of context In the activation of early hypotheses, groups of 
family physicians and graduate or near-graduate medical students were presented with 32 
short case-histories, each containing a picture of a patient, the previous disease history and 
the present complaint. After reading each case, the subject was asked to formulate a diagnosis. 
Exposure times were fixed. It was expected that differences In the number of correct diagnoses 
would emerge between the two groups and that, according to Feltovlch s theory, the 
experienced physicians process relevant contextual information more extensively than novices. 
If the more experienced group uses certain contextual Information provided by the picture and 
previous disease history in a more elaborative way, leaving memory traces of greater 
distinctiveness, this would show in the information recalled, because more elaborative 
processing of information produces better recall (Anderson & Reder, 1979). Therefore subjects 
were asked to recall all the information they considered to be important to the case. 
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METHOD 

Subjects; Subjects were 35 physicians and graduate medical students. The group of experts 
consisted of 18 family physicians who on the average had 1 1.1 + 7.7 years of experience in 
health care. Average age was 38.2 + 9.1 years. The novices were 12 final year medical students 
who were about to graduate and 5 physicians who graduated less than one month prior to the 
experiment. Their average age was 25.5 + 3.3 years. 

Materials: It was already outlined that enabling conditions are not explicitly available in the 
early moments of an encounter. We have tried to simulate this condition by presenting the 
information with a portrait and medical card of the patient. In doing so, we were able to give 
information about sex, age, profession, previous disease history, etc. In an implicit way. The 
portrait, patient chart and the presenting complaint were presented on three separate slides. 
The stimulus materials consisted of 32 case histories selected from a larger set of 
information about patients available at the University of Llmburg Medical School. The cases 
were selected In such a way that contextual information could play an important role in the 
interpretation of the complaint. The pictures were selected from a set of 110 portraits 
specially produced for the experiment. They were only to convey information concerning age and 
sex of the patient. Therefore, only pictures were utilized on which the person showed a neutral 
facial expression and no signs of any disease (like exophthalmus or a drooping corner of the 
mouth). Previous disease history was typed on a patient chart. In addition, this chart contained 
Information about the patient's profession, marital status, medication, family history of 
diseases and risk factors. The Items were selected in such a way that data Included both 
relevant and Irrelevant information as related to the presenting complaint. The presenting 
complaint consisted of one or two sentences as expressed by the patient, for instance: "I have a 
cold fever for already two days, doctor. I sometimes lie down shaking In my bed". The 
complaints covered all organ systems. 

Procedure: The 32 cases were presented In a standardised way. First, the subject was 
exposed to the portrait for about 4 seconds, then the patient chart appeared on the screen. 
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followed by the presenting complaint. Exposure times were fixed, but varied between 3 and 42 
seconds for the chart, and between 3 and 9 seconds for the complaint, because some patient 
charts and complaints contained more information than others. The times were established in 
several pilots with experts and novices. Mean exposure time per case was 32 seconds. Finally, 
a black slide was projected for about 15 seconds. During that Interval the subject was asked to 
state the most likely diagnosis, given the information presented. 

After an instruction session, the 32 cases were presented In two series of 16 each. After both 
series, each of the 16 presenting complaints was read back to the subject, together with the 
tentative diagnosis the subject had generated. Subsequently the experimenter asked the subject 
to recall which information embedded in the case gave rise to the particular hypothesis. All 
responses were audlotaped. Presentation-order of the two series was varied systematically 
over subjects. 

Scoring; A verbatim transcription was produced from the audiotaped responses. For each 
subject and for each case It was determined whether the hypothesis generated was correct and 
how much Information was retained. 

Diagnostic accuracy. As a criterion for the accuracy of the hypothesis the actual diagnosis 
of the particular patient was used. Two judges compared the statement of the subject with the 
actual diagnosis. Interrater agreement for this task was 95.4%. 

Accuracy and completeness of recalled information, in order to judge how much 
information about each patient was recalled, the information on the patient chart was 
segmented into Information-units. An Information-unit was defined as a statement containing 
one singular fact or idea. The portrait of the patient and the name and birth-date on his chart 
were considered to contain two units of information: sex and age. For each Information-unit it 
was determined whether It was relevant to the correct diagnosis or not. 
The segmented case-information was compared with the verbatim transcripts of the subject s 
responses. 

The accuracy and completeness of recalled information was determined as follows. Of every 
unit recalled it was decided whether it could be considered literal or paraphrased recall, 
partial or Inferred recall, or a summary of the contextual Information presented. 
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These responses were weighted. Each literal or paraphrased information-unit recalled was 
scored as 3 points. Partial or inferred Information units were scored 2 points, whereas a 
summary was scored as 1 point. For Instance; the patient who presented with the complaint of 
shaking chills was a male born October the 15*1931. He had had a hematuria, after which 
carcinoma of the bladder was diagnosed. This diagnosis led to a total cystectomy with 
diversion of the urine by uretero-ileostomy. Examples of literal or paraphrased recall by 
subjects are: "fifty-five years old..." and "they removed the bladder..." Examples of partial or 
inferred statements are: "older patient..." and "bladder operation...". An example of a summary is: 
"there was something in the chart about urinary tract problems...". 

In this way complete answers of subjects could be scored as follows. With regard to the above 
described patient-case, subject 8 motivated his hypothesis in these statements: "this man has 

had a total cystectomy, then you get there will be an isolated ileal loop with implanted 

ureters. Well in such a case you often see ascending infections". This subject scored a tote I of 
9 points, that is to say, 3 points for "man", 3 points for "total cystectomy" and 3 points for 
"isolated ileal loop with implanted ureters". Subject 1 was much shorter in his motivation: "I 
believe there was something on the chart about urinary tract problems, that's why an infection 
came high on my list. Beside that, fever can be very high then". This subject scored 1 point for 
his summary as already exemplified above. 

Interrater agreement of two judges for this task was 93.4JS on one fifth of the material. The 
remaining responses were scored by one judge. 

Group differences in diagnostic accuracy and in recall of contextual information were analysed 
by means of a one-way-analysis-of-variance. 



9 

ERIC 



Contextual Factors 8 

RESULTS AND DISCUSSION 

Table 1 shows the average number of correct hypotheses produced by experts and novices. 

Table 1. Average number of correct hypotheses, produced by experts and novices (with standard 

deviations) 

N Mean SD 

Experts 18 12.11 2.52 

Novices 17 8.88 2.12 



The observed difference is statistically significant: F( 1,33)= 16.75, p=.0003. These results 
indicate that experts produce more accurate first hypotheses than novices do, given a very 
restricted amount of information about a patient. It is interesting to observe how well experts, 
in the absence of additional data such as the results of physical examination or laboratory 
tests, are able to solve a diagnostic problem. Even before hi story- taking they are already on the 
right track in almost 40% of the cases. 

The central thesis of this research Is that experts are better able to utilize the available 
information In an information-restricted environment than novices do, even if this information 
is not overt related to the complaint at hand. Recall of information summarized over two series 
of trials are shown in Table 2. For the purpose of this study it was Important to discriminate 
between recall of relevant, and recall of irrelevant information-units. 
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TABLE 2: Average recall scores by experts and novices (standard deviations between brackets) 
Experts Novices F P 

216.05(52.84) 163.35(48.56) 9.41 .004 
180.33(40.99) 136.24(36.50) 11.25 .002 
35.74(14.68) 27.11(13.02) 3.35 .076 



Total recalled 
Relevant 
Irrelevant 



The recall of information-units by the experienced physicians exceeds that of the noviced, 
with the exception of recall of irrelevant information-units. The effect Is most striking for the 
information that was relevant for a correct interpretation of the complaint. This differential 
retrieval of information indicates that the representational format used by experts to 
understand a patient's problem (e.g. illness scripts: Feltovlch & Barrows, 1984) must include 
structures for storing relevant context. 

The data presented do not automatically imply that experts produce better hypotheses 
because they use contextual information in a more elaborative way. Whether the scores are 
the retinue of a causal relationship remains to be questioned. An alternative explanation for 
the superior performance as shown in Table 1 might be that the experts have better lists of 
possible diagnoses stored in memory, that are activated by the presenting complaint alone. This 
would imply that their guesses are better, not so much because contextual factors are critical 
to their performance, but because they have developed more appropriate lists of diagnoses in 
relation to certain sets of complaints. However, this alternative explanation cannot account for 
the observed expert correlations in as shown in Table 3, because recall of complaint 
information is not included in the scores. 
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TABLE 3: Product-moment correlations between number of correct hypotheses and recall 

measures for each group 

correct hypotheses 
Experts Novices 

Total Recall .54* 
Relevant Recall .63** 
Irrelevant Recall .17 



.12 
.15 
.03 



*p<.05 **p<.oi 

Table 3 shows high correlations between total number of accurate hypotheses and recall of 
contextual information for experts. Since correlations between recall and problem solving 
measures are absent in the novice group, the previously mentioned alternative explanation 
might provide an accurate model for the well -performing novice. In our view the only 
reasonable explanation for the expert-novice differences in the observed correlations is an 
expert-novice difference in elaboration on contextual information. Only a difference in 
reconstructive processes during retrieval, not related to a better elaboration, could also have 
an effect on recall. However, such a difference cannot cause a better diagnostic performance. 
The knowledge activated by the contextual information in the 32 cases covers a broad range of 
subject-matters; from disease-distributions to side-effects of drugs. Generally, in the 
training of medical students to become diagnosticians much attention is paid to complaint- 
exploration and physical examination. This is mainly due to the fact that most of the time 
training occurs in clinical settings like hospital wards. Moreover in medical education 
emphasis lies on anatomic and pathophysiological knowledge in order to let the students 
understand the patients symptoms and signs. The results of this experiment, however, indicate 



that in a critical phase of the diagnostic process another kind of knowledge is used namely that 
needed to understand the context of a patient. 

This finding needs further exploration. For instance, there are good reasons to believe that the 
importance of contextual information in medical problem solving varies with the kind of 
context of the diagnostician (e.g. specialist versus general practitioner). 

In conclusion, the data provide support for the notion that experienced physicians make an 
extensive use of contextual information while attempting to solve diagnostic problems, at 
least when no additional information regarding the present condition of the patient is 
available. They confirm the assertion of Feltovich and Barrows (1984) that enabling conditions 
form an essential part in the construction of a mental representation of the patient's problem 
and help the physician to reduce the number of alternatives he has to take into consideration. 
The results also indicate that this skill is acquired only through years of clinical practice. 
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